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Field of the Invention 

This invention is related to the field of digital signal 
10 processing, and more particularly to a data format and to a system 
for decoding MPEG compatible data incorporating Internet 

information. 

Background of the Invention 

1 5 Home entertainment systems which combine -Personal 

Computer and television functions (PC/TV systems), are 
fnTealgly becoming, generic. User interactive, mulnple source 
and muulple destination communication devices. Such systems are 
2 0requi?ed to communicate in different data formats between 
mulul locations for a variety of applications in response to User 
reques- For example, a PC/TV system may receive data rom 
satellite or terrestrial sources comprising H.gh ™™ lon 
revision (HDTV) broadcasts, Multi-poin, Microwave Dts— 
25 System (MMDS) broadcasts and Digital Video Broadcasts (DVB). A 
PC/TV system may also receive and transmit data vta telephone 
^r*e ImeTnet) and coaxial lines (e.g. cable TV) and from both 
emote and local sources such as Digital Video Disk (DVD), CDROM 
VHS and Digital VHS (DVHS™) type players, PCs, and many other 
3 0 types of sources^ ^ ^ ^ 

e eneric PC/TV entertainment system. Specifically, problems arise 
^supporting communication between multiple data sources and 
in Zceslg data from the multiple sources. A problem .. also 
3 5 presented oy the need «o devise a User interface for such a system 
Z supports complex User interactive tasks whilst proving a 
lilple command interface suitable for the general pubhe. A 
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PC/TV system User interface, for example, should allow a User to 
view a selected program and permit User operation of functions such 
as Email, telephone, Internet access, fax and video-phone functions. 
Such applications, require communication between a PC/TV unit and a 
variety of both remote sources e.g. a satellite service provider, and 
local sources e.g. a DVD storage device. Further, a PC/TV needs to 
process and decode data in different data formats from different 
devices and display received data to the User. These problems are 
addressed by a system according to the present invention. 

A prior art reference includes U.S. Patent 5,544,161 which 
discloses a system for conveying MPEG compatible data in ATM 
(Asynchronous Transfer Mode) protocol. 

Summary of the Invention 

The inventors have hereby recognized that a Program 
Guide type of User interface advantageously provides a simple, easy 
to use interface for User operation of functions such as Email, 
telephone, Internet access, fax, home control, and video-phone 
functions. The use of a Program Guide for such functions also 
advantageously provides a single User interface for User operation of 
multiple devices and associated functions. 

The inventors have further recognized that Internet 
information requested via a program guide or other type of User 
interface may be advantageously conveyed in a composite data 
format including MPEG compatible video data. They have further 
recognized that it is desirable for a video decoder system to be 
capable of decoding such a composite data format. 
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A disclosed data format includes MPEG compatible video 
data and Internet image information suitable for conveying 
packetized video program data and associated web image 
5 information. The data format comprises MPEG system data that 

includes a first transport packet with a header containing a first data 
identifier (PID) for identifying the video program data in a payload of 
the first transport packet. The MPEG system data also includes a 
second transport packet containing a second data identifier (PID) for 
10 identifying Internet information contained in a payload of the second 
transport packet. The MPEG system data also includes ancillary data 
supporting the identification and 
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decoding of the Internet information and containing the second 

data identifier. 

In a feature of the invention, a method for decoding 
data coded in the composite data format involves applying the 
5 ancillary data in identifying, decoding and assembling the Internet 
information. 

In another feature of the invention, a method for 
decoding data coded in the composite data format involves 
synchronizing the Internet information with the video program 

1 0 using the ancillary data. 

Brief Description of the Drawings 

In the drawing: 

1 5 Figure 1 shows a home entertainment decoder system, 

according to the invention, for communicating with a plurality of 
sources and for processing program guide information and 
program content data. 

2 0 Figure 2 shows an exemplary Program Guide display 

format according to the invention. 

Figure 3 shows a data format, according to the 
invention, for packetizing Internet data for transmission within an 

2 5 MPEG compatible datastream. 

Figure 4 shows a flowchart, according to the invention, 
for receiving program guide information including User selection 
options and processing the received information to form a 

3 0 program guide supporting User selectable communication 

functions. 

Figure 5 shows a flowchart, according to the invention, 
for a process to form a program guide in a video decoder from 
3 5 information received from a plurality of sources. 
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Figure 6 shows a flowchart, according to the invention, 
for a process to automatically acquire and collate program guide 
information from a plurality of sources and to form a program 
guide for display. 

Figure 7 shows a black and white representation of an 
exemplary color Program Guide display format, according to the 
invention, showing menu options permitting a User to collate 
programs by source and color attribute. 

Figure 8 shows a flowchart, according to the invention, 
for a process to acquire information requested by a video receiver 
User via a selection made from options on a displayed program 
guide. 

Figure 9 shows a flowchart, according to the invention, 
for a process to decode input Internet information and 
compressed video data and to form a composite video output for 
display. 

Detailed Description of the Drawings 

Figure 1 shows a home entertainment decoder system, 
according to the invention, for communicating with multiple 

2 5 sources to acquire program guide information and program 

content data. The decoder system forms a program guide for 
display from information acquired and collated automat.cally from 
multiple sources. A User may initiate Email, telephone, Internet 
access, fax and video-phone functions via selection of options from 

3 0 the displayed program guide. The decoder system adaptively 

decodes a datastream encoded to the MPEG standard incorporating 
compressed program content data and Internet data 
representative of, for example, Hypertext Mark-up Language 
(HTML). The decoded data is formatted either as a composite 
3 5 video image or as separate video images for display. 

The MPEG2 (Moving Pictures Expert Group) image 
encoding standard, hereinafter referred to as the "MPEG standard" 
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is comprised of a system encoding section (ISO/IEC 13818-1, 10th 
June 1994) and a video encoding section (ISO/IEC 13818-2, 20th 
January 1995), hereinafter referred to as . the "MPEG systems 
standard" and "MPEG video standard" respectively. 

5 Although the disclosed system is described in the 

context of a system for receiving an MPEG compatible signal, it is 
exemplary only. The principles of the invention may be applied to 
systems in which the types of transmission channels and 
communication protocols may vary, or to systems in which the 

1 0 coding type may vary. Such systems may include, for example, 
non-MPEG compatible systems, involving other types of encoded 
datastreams and other methods of conveying Program Guide 
information. Further, although the disclosed system is described as 
processing broadcast programs, this is exemplary only. The term 

1 5 'program' is used to represent any form of data such as telephone 

messages, computer programs, Internet data or- other 
communications, for example. 

In overview, in the video receiver system of Figure 1, 
a carrier modulated with video data is received from a broadcast 

2 0 satellite by antenna 10 and processed by unit 15. The resultant 

digital output signal is demodulated by demodulator 20 and 
decoded and error corrected by decoder 30. In this exemplary 
embodiment, the demodulated and decoded output from unit 30 is 
in the form of an MPEG compatible transport stream containing 

2 5 MPEG compressed video data and Internet data encoded in HTML. 

Video data encoded to the MPEG standard is in the form of a 
packetized datastream which typically includes the data content of 
many program channels (e.g. content corresponding to cable 

television channels 1-125). 

3 0 The MPEG compatible transport stream is provided to 

processor 25 which is responsive to commands from remote 
control unit 125. Processor 25 communicates with other data 
sources including storage device 90 and either Internet data 
server 83 or Internet connection service 87 (e.g. America On- 
3 5 line™). Program guide information is acquired and collated 
automatically from the input transport stream and storage device 
90 and either server 83 or connection service 87. The acquired 
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information is merged by processor 25 to form a program gui de 
for display incorporating selection options allowing a User^to 
late Email, telephone, Internet access, . fax and videophone 
functions among others, for example. The program gu.de for 
5 s also formed to indude non-displayed information from 

muUiple sources for identifying and assembhng u .dividual d a 
nackets that constitute the programs that are selected for U er 
viewing or listening. This information is used in recovenng the 
content of selected programs from multiple purees 

Processor 25 adaptively decompresses the input 
compressed video data and decodes the HTML Internet to 
P 3 a formatted composite video image for display on urn 7* 
Processor 25 also provides encoded data outputs for storage on 
™ medium 105 via storage device 90, and provides encoded 
15 data to ler devices (not shown to simplify drawing) v.a modem 
SO and -lephone d ,ines.^^ ei ^ ^ 

Email telephone, Internet access, fax and video-phone funcuons 
!» m d vielg and storage of selected programs o, 
2 0 options from the displayed program gu.de. A Use 
flayed option with a cursor « fg remc. contr o ui. 12, 

o°pera°t on me — „? processor 25 and responds to remote 

2 5 contrTunU 125 commands using a bi-directional data and control 

signal bus C. Controller 115 controls the functions of ind.v.dual 
elem nts within processor 25 by setting control reg.ster values 

sr these *l- i c *rs£i£ 

30 IT si IgHI Lt 90 is a DVD type device 

3 0 1U3 via siuiage . containing a 

and medium 105 comprises a multi-disK stacK un 

nluralitv of disks in this exemplary embodiment. 

p mrahty <* p . gure , in detail , . carrier modulated 

with video data received by antenna !0 i s converted to 
3 5 form and processed by . input processor 15. Processor 13 »» c 

al frequency (RF) tuner and intermediate frequency (IF mixe 
and amplification stages for down-converting the input video 
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signal to a lower frequency band. Processor 15 also includes an 
analog to digital converter for digitizing the down-converted 
signal to produce a signal suitable for further processing. The 
resultant digital output signal is demodulated by demodulator 20 
5 and decoded and error-corrected by decoder 30. The output from 
decoder 30 is further processed by unit 45 of processor 25. 

The data provided to unit 45 is in the form of an MPEG 
compliant packetized transport datastream as defined in MPEG 
systems standard section 2.4 and includes program guide 
1 0 information and the data content of one or more program 
channels. Processor 25 acquires and collates program guide 
information automatically from the input transport stream and 
storage device 90 and either Internet data server 83 or Internet 
connection service 87 (e.g. America On-line™). The individual 
1 5 packets that comprise either particular program channel content, 
or Program Guide information, are identified by their Packet 
Identifiers (PIDs) contained within header information. 

In response to control signal C, unit 45 selects either 
the transport stream from unit 30, or in a playback mode a 
2 0 datastream retrieved from storage device 90 via store interface 
95 In normal, non-playback operation, individual packets that 
comprise program guide information are identified by unit 45 in 
the transport stream from unit 30 and assembled using PIDs that 
are predetermined and stored in internal memory of controller 

2 5 115 Unit 45 matches the PIDs of incoming packets in the 

transport datastream from decoder 30 with PID values pre-loaded 
in control registers within unit 45 by controller 115. Therefore, 
satellite broadcast program guide (SPG) information packets in the 
transport stream are identified by unit 45 without additional PID 

3 0 information. Controller 115 acquires a full SPG by accessing and 

assembling the SPG packets that are identified and captured by 

unit 45. , K ■ 

The PID information that enables controller 115, in 

conjunction with unit 45, to identify data packets that comprise 
3 5 individual programs is termed a program or channel map. The 
program map associates PIDs with individual packetized 
datastreams that constitute a program and is part of the SPG. The 
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SPG may also contain additional information, e.g. packet assembly 
information, supporting the recovery of program guide data from 
the input transport stream. 

Controller 115 also configures unit 45 using Control 
5 signal C to select the data packets comprising the DVD program 
guide (DPG) information derived from DVD storage device 90 via 
interface 95. Unit 45 matches the PIDs of incoming packets in the 
packet datastream from interface 95 with PID values pre-loaded 
in control registers within unit 45 by controller 115. Controller 
10 115 acquires a full DPG by accessing and assembling the DPG 
packets that are identified and captured by unit 45. 

A similar process is followed in acquiring Internet 
program guide (IPG) information from Internet server 83 or from 
Internet connection service 87 by telephone line communication 
1 5 via modem 80. In acquiring the IPG, data identifiers analogous to 
or the same as the PIDs used in acquiring the SPG or DPG are 
matched with identifiers pre-loaded by controller 115. The 
matching function may be performed either within unit 80 or by 
controller 115. The process for acquiring program guide 
2 0 information is explained in greater detail in the description of 
Figures 4-6. 

The SPG, DPG and IPG information acquired by 
controller 115 is merged by processor 25 to form a program guide 
for display incorporating selection options allowing a User to 

2 5 initiate Email, telephone, Internet access, fax and video-phone 

functions. In merging the data, the acquired SPG, DPG and IPG 
information is collated into two levels of hierarchy termed theme 
and topic and redundant program guide information is eliminated. 
A theme, for example, may include categories such as movies, 

3 0 sports, weather, arts, documentaries, news, etc. A topic is a 

category within a theme, for example, a movie theme category 
may have topics including, comedy, thriller, horror, science fiction, 
romance, adult entertainment, etc. Different or additional levels of 
hierarchy may be used in collating the acquired SPG, DPG and IPG 
3 5 information involving criteria such as viewer age suitability, the 
period setting e.g. contemporary or historical, reviewer ratings etc. 
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A number of problems occur in collating the SPG, DPG 
and IPG information. The SPG, DPG and IPG information, being 
from different sources, may categorize programs by different 
themes and topics and may use a different sorting hierarchy. 
5 Further, the SPG, DPG and IPG data may use conflicting, redundant 
or incompatible PIDs or other data identifiers for identifying 
individual data elements that comprise particular program 
channel or guide content. Controller 115 creates a master set of 
themes and topics mapping the SPG, DPG and IPG information to a 
1 0 corresponding category in the master set. For example, the master 
set may contain an encompassing topic-theme of 'movies-comedy' 
to which is mapped both, a) the DPG topic-theme of 'movie- 
parodies', and b) the IPG Internet topic-theme of 'movie-synopsis 
index' (an index of web pages). In collating the guide information, 
15 controller 115 determines the topic and theme categories of the 
data in the SPG, DPG and IPG from pre-determined theme-topic 
codes. The SPG, DPG and IPG theme arid topic codes together with 
equivalence mapping information for converting codes of one 
service provider, to those of another, are acquired from the SPG or 
2 0 User defined data input to processor 25 or are pre-stored in 
internal memory within controller 115. Controller 115 assembles 
the collated SPG, DPG and IPG packets according to the categories 
defined by the master set of themes and topics. The assembled 
collated SPG, DPG and IPG data is stored by controller 115 in 

2 5 internal memory for use in forming a composite program guide for 

display. 

Controller 115 also forms MPEG compatible non- 
displayed information, associated with the displayed composite 
program guide. The non-displayed information includes composite 

3 0 program map, conditional access and network information. This 

non-displayed information is required to support identification, 
assembly and decoding of the packets comprising the composite 
program guide and the associated program content. The 
conditional access information governs access to programs based 
3 5 upon User entitlement. .The network information defines physical 
network parameters such as satellite transmission channel 
frequencies and transponder channels, for example. 
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Controller 115 forms composite program map 
information for the composite program guide that maps the SPG, 
DPG and IPG themes and topics to the master set of themes and 
topics. The composite program map also associates data identifiers 
5 (e.g. PID values) with individual packetized datastreams that 
constitute programs listed in the composite program guide. The 
composite program map information may also include a dedicated 
indicator to denote that a particular program has associated 
Internet web page information available. In forming composite 
10 program map information, controller 115 examines the SPG, DPG 
and IPG information for data identifier omissions, redundancies 
and conflicts. Controller 115 re-numbers existing data identifiers 
and creates new data identifiers as required to produce MPEG 
compatible program map information for both the collated 
1 5 program guide information and the associated program content. 

Controller 115 also forms MPEG compatible conditional 
access information and network information from corresponding 
information received in the SPG, DPG and IPG data. The conditional 
access information includes data such as encryption codes that 
2 0 govern access to programs that are dependent upon user 
entitlement. The network information defines physical network 
parameters such as satellite transmission channel frequencies, 
transponder channels, and Internet access parameters associated 
with the composite program guide for display. The composite 

2 5 program map, conditional access and network information 

supports assembly, decoding and display of the program content 
available from the satellite, storage and Internet server sources as 
well as the composite program guide. 

Controller 115 also includes additional linking data in 

3 0 the non-displayed composite program map, conditional access and 

network information. The additional linking data supports the 
User selectable functions such as Email, telephone, Internet access, 
fax video-phone, channel tuning, program recording and home 
control functions. Home control functions may support User 
3 5 control of lighting air conditioning, heating and burglar alarm 
operation, for example. The additional linking data associates 
access data, both with data identifiers (e.g. PIDs) of requested 
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data, and with User selectable menu options in the displayed 
composite program guide. The access data includes identification 
codes, communication protocol codes, conditional access codes, 
Internet access codes e.g. URL codes for web page and Email access 
5 as well as telephone and fax numbers. The access data enables 
processor 25 to communicate with external devices. 

In order to generate the composite program guide for 
display, controller 115 retrieves the assembled and collated SPG, 
DPG and IPG packet data previously stored in internal memory. 
10 Controller 115 both converts the retrieved data to pixel data and 
formats the converted data for storage in a pixel representative 
memory map contained in display processor 70. In addition, 
controller 115 creates User selectable menu options (for User 
selection of Email, fax etc.) in pixel data form for incorporation in 
15 the unit 70 pixel memory map. Controller 115 stores the 
formatted SPG, DPG and IPG pixel data together with the 
generated option pixel data in the unit 70 pixel map to form the 
composite program guide for display. 

The composite program guide, combining the SPG, DPG 
2 0 and IPG data and User selectable options, is formatted to be 
compatible with the linking data in the non-displayed program 
guide information. For example, a User selectable option in the 
displayed guide (e.g. for a telephone function) is associated, both 
with access data for the function (e.g. a telephone number), and 

2 5 with data identifiers (e.g. PIDs for data packets of transmitted and 

received telephone messages). 

Display processor 70 converts the stored and 
formatted program guide data into conventional luminance and 
chrominance components. The luminance and chrominance 

3 0 components, together with timing data provided by controller 115, 

are processed by processor 70 in a conventional manner to form 
an NTSC signal for display on NTSC compatible image reproduction 
device 75 e.g. a television or video monitor. Processor 70 may, in 
another embodiment, provide RGB output data for display by an 
3 5 RGB compatible reproduction device or a high definition television. 

Controller 115, in conjunction with processor 55, forms 
an MPEG compatible datastream of compressed video and audio 
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data containing the composite program guide and program 
content. Processor 55 separates system information including 
timing, error and synchronization information from the 
datastream from unit 45 and diverts it for use by controller 115. 
5 Controller 115 applies the system information in controlling 
processor 55 to provide re-synchronized, collated program and 
composite program guide data together with synchronization and 
error indication information for use in subsequent program 
content data decompression by decoder 65. The composite 
1 0 program guide data provided by processor 55 combines the SPG, 
DPG and IPG data and User selectable options in a manner 
compatible with the program map linking information as 
previously described. The program content data provided by 
processor 55 is decompressed by MPEG decoder 65 to provide 
15 video pixel data and audio output data to processor 70. Display 
processor 70 processes the video pixel data and audio output data 
to be suitable for reproduction by unit 75 as previously described. 

Figure 2 shows an exemplary displayed composite 
program guide. Menu features 833-850 show program services 
2 0 and functions. Menu features 833 and 835 show program channels 
which may be invoked and displayed upon highlighting of the 
corresponding menu feature using a User selection control such as 
a remote control directed cursor. If a User highlights feature 833 
with the cursor, i.e. selects menu option 833, for example, 

2 5 processor 25 tunes to television channel 107 (A&E™) and displays 

it on unit 75 using the composite non-displayed program guide 
information to determine the function activated and associated 
tuning information (channel frequencies etc.). Similarly, if a User 
highlights feature 837, processor 25 tunes to receive radio channel 

3 0 FM 13 for audio output on unit 75. Upon highlighting features 

840-850, corresponding to Email, telephone, fax and DVD 
record/playback functions respectively, additional function 
specific menus become displayed providing further User selection 
options. Such function specific menus may provide a User with the 
3 5 ability to edit, store and delete access information including, for 
example, telephone numbers, Email addresses, Internet addresses, 
credit card numbers and additional conditional access data. 
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If a User highlights menu features 860-870 specific 
function tasks are initiated. Upon a User highlighting feature 860, 
for example, a further menu is displayed prompting the User to 
enter an Email message to be sent to John Smith at his home via 
5 an Internet Email address retrieved from the composite non- 
displayed program guide information. Similarly, in response to a 
User highlighting feature 865, for example, controller 115 (Figure 
1) retrieves Jean's home telephone number from internal memory 
using the composite non-displayed program guide information. 
10 Further, controller 115 establishes telephone communication, via 
modem 80, with Jean at her home by dialing Jean's home 
telephone number. Also, upon a User highlighting feature 870, for 
example, the movie 'Star Wars™' is re-played from DVD unit 90 
(Figure 1). The communication between processor 25 and DVD 
1 5 unit 90 is established with access data retrieved from memory 
within controller 115 using the composite non-displayed program 

guide information. 

In addition, the program guide may include one or 
more icons, such as icon 873 (Figure 2) depicting a home for 
2 0 example, to indicate whether Internet information such as a home 
page' is available for a particular program. Further, icon 873, in 
Figure 2 may change color e.g. from white to green to indicate 
Internet information is available for a particular program. 
Alternatively, icon 873 may be a pop-up icon which appears as an 

2 5 indication that Internet information is available for a particular 

program. The displayed program guide may be formed to enable a 
User to request Internet information on a particular program by 
highlighting the program followed by highlighting the pop-up icon, 
for example. Home page icons that are present when Internet 

3 0 information is available may also located within or adjacent to 

program icons, e.g. icon 859. Further, the home page icon may 
change color upon User selection of the information by 

highlighting the home page icon. 

Although Internet information may be retrieved upon 
3 5 highlighting a web page icon such as icon 857, in Figure 2 this is 
exemplary only. Retrieval of Internet information may also be 
initiated in a variety of other ways. A user may retrieve available 
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Internet information relating to a program, by highlighting^ 
program icon such as 'Seinfeld™- using icon 859 followed by 
highlighting Internet icon 810, for example. 

Upon User selection of a composite program guide 
5 function, controller 115 (Figure 1) determines and applies the 
access data associated with both request and response operations 
involved in the selected function. Controller 115 determines the 
access data required for the request and response operations from 
the previously formed non-displayed composite program map, 
1 0 conditional access and network information. 

Processor 25 acquires information, such as web page 

~ f« a TTcer reauest in a number of ways, 
data, in response to a user reque^i, 

including, for example: 

15 a ) In a first configuration, processor 25 accesses web 

page data that is continuously transmitted in the satellite 
broadcast received from decoder 30, 

b) In a second configuration, processor 25 acquires 
2 0 web page data that is transmitted in the satellite broadcast 

received from decoder 30 in response to an information request to 
a service provider via telephone line, and 

c) In a third configuration, processor 25 both requests 

2 5 information and receives response information on the same 

communication link e.g. on the telephone link to server 83. 

v ^t rnnfig y™tinn of Processor 25 

3 0 In the first configuration, web page data selected by a 

User via the displayed program guide is continuously transmitted 
within the transport datastream input to processor 25 from 
decoder 30. In this configuration, the composite program map 
information contains predetermined PID values for the requested 
3 5 web page. Consequently,, the requested web page data is retrieved 
from the input transport stream using the composite program map 
information without the need for controller 115 to communicate 
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with a service provider via modem 80. In this configuration, the 
web information available to a User is limited to the specific 
information that a service provider continuously transmits to 
processor 25 using dedicated transmission bandwidth. However, 
5 this configuration is simple and offers quick web page access 
(since the web data may be stored locally at the encoder by the 
service provider). This configuration also minimizes the service 
provider burden of data acquisition and eliminates the need for 
processor 25 to communicate web page request information to a 

1 0 service provider. 

The web page information, encoded in HTML™ and 
Java™, is packetized and formatted in an encoder by the service 
provider for incorporation into the MPEG transport datastream for 
transmission. Figure 3 shows a data format, according to the 
15 invention, for packetizing Internet data for transmission within an 
MPEG compatible datastream. A sequence of HTML™/Java™ web 
pages (705-720 of Figure 3) is formed by the encoder into non- 
compressed packets with ancillary information and HTML™/Java™ 
data (725). The ancillary information includes PID values, a packet 

2 0 count value, and individual web page size (e.g. number of bytes or 

image pixel size). The ancillary information may also include: a 
flag to indicate that a particular transport packet contains web 
page information; a start code; a web page identification code; an 
error correction code and a timing parameter suitable for 

2 5 synchronizing web page Internet information with an associated 

video or audio program. The web page flag may be used to 
facilitate identification and separation of the web page 
information from the MPEG compressed video data. Further, one 
or more elements of the web page ancillary data may be included 

3 0 within User definable elements of the MPEG transport syntax, such 

as within the transport header, or it may be included along with 
the web page information in the MPEG transport data payload. 

The response web page information, including a 
predetermined PID value, is incorporated in the MPEG compatible 
3 5 transport datastream input to processor 25 (Figure 1) by the 
service provider. The response channel predetermined PID value 
is also prestored in controller 115 internal memory and 
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Ze a high definition tension (HDTV) signa. encoded 

' thp n.oira/ Television Standard for HDTV 

5 :l „ o $.12 »*. -p- ty the un r states 

A^a ced Television "Urns Committee (ATSC). In such a case 
one or more specific pre-determined program channel may be 
designated to convey web page response information 
10 PID values of the program channels may be conveyed in a CIT 
the composite program map information, for example 

Controller 115 derives the predetermined PID value 

incoming p* pre-loaded in the 

datastream from decoder 3C .with he PID P m 

» s.T=r -~ »r" 

• • ,„„, and synchronization information from the 
tuning, erro and sy ^ ^ u5 

"con" U5 appues the system information in directing 
£2TS5 to provide re-synchronized web 

processor 60. Processor ou uc nrnvides we b page image 

115 Although processor 60 is shown as a separate processor in 
F Le r U may also be implemented within the software o 
Roller' 15 Processor 60 comprises the processing functions of 

Navigator™, for example. 
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Program content data e.g. a User selected television 
program channel, within the MPEG compatible transport 
datastream from decoder 30, is also decoded into pixel 
representative data by processor 25 in addition to the web page 
5 response data. For this purpose, controller 115 determines the PID 
value of the selected television channel program from the 
composite program map information and loads it into control 
registers within unit 45. Unit 45 matches the PIDs of incoming 
packets of selected television channel program data in the 
1 0 transport datastream from decoder 30 with the PID value pre- 
loaded in the unit 45 control registers. The packets with PID 
values matching the pre-determined selected television channel 
program PID value are identified and provided by unit 45 to a 
dedicated application buffer in decoder 55. 
15 As previously described for the web page response 

data processor 55 derives system information including timing, 
error and synchronization information from the datastream from 
unit 45 and provides it to controller 115. Controller 115 applies 
the system information in directing processor 55 to provide re- 
2 0 synchronized program channel content data from the application 
buffer in decoder 55 to MPEG decoder 65. The MPEG compatible 
program channel content data is decompressed by MPEG decoder 
65 to provide video pixel data and audio output data to display 
processor 70. The video pixel data from decoder 65 is stored m a 

2 5 pixel representative memory contained in display processor 70. 

Controller 115 retrieves the web page image pixel data 
from internal memory and formats it for storage as an overlay in 
the pixel representative memory contained in display processor 
70. As such a merged display combining the web page response 

3 0 data and program channel content is formed within the pixel 

memory of processor 70. The relative proportions and sizes of the 
web page image and channel content image within the resultant 
composite image, formed in the processor 70 pixel memory, may 
be varied by controller 115 in response to internal programmed 
3 5 instruction or external, e.g. User, command. The relative 
proportions may be varied so that either the web page image or 
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channel content image contributes between 0-100% of the 

resultant composite image. 

In order to adjust the proportion of the compos.te 
image contributed by the web page data, controller 115 scales the 
5 web page image pixel data by a conventional interpolation 
process. Alternatively, the data may be scaled by a separate vuteo 
processing integrated circuit operating in conjunction with 
controller 115. The resulting scaled web page image is stored 
within the processor 70 pixel memory by controller 115. The 
1 0 stored web page image represents an overlay image with respect 
to the program channel content image. In order to adjust the 
location of the web page image data within the compos.te .mage, 
controller 115 determines the addresses corresponding to the 
desired memory locations from internal memory map data and 
1 5 stores the scaled web page image data at the desired memory 
locations. As previously explained, display processor 70 converts 
the formatted composite image data into conventional luminance 
and chrominance components. Further, processor 70 processes the 
luminance and chrominance components, together with timing 
2 0 data provided by controller 115, in a conventual manner to form 
an NTSC signal for display on NTSC compatible image reproduction 
device 75. 

CmnH Pft nfi r r a tinn of Processor 25 

In the second configuration, processor 25 of Figure 1 
acquires web page data that is transmitted in 
broadcast received from decoder 30 in response to an ."formation 
request to a service provider via a telephone line. Upon a User 
3 0 requesting web page data, e.g. by highlighting web page icon 857 
XL 8 2, corTtroller 115 (Figure 1) uses the request access data 
(telephone number, requested web page identifier code 
and conditional access data) to establish communication with the 
satellite broadcast service provider. 

prior to establishing communication with the service 
provider, controller 115 determines from conditional access 

• h»,i,»r access to the requested web page 

information whether access to i 
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information is authorized. Authorization may be determined 
directly from the program guide conditional access information or 
indirectly from a User entitlement device such as a smart card 
(not shown to simplify the drawing) within processor 25. If access 

5 is authorized, controller 115 establishes telephone communication 
with the satellite broadcast service provider by dialing a 
telephone number via modem 80 and transmitting request 
information e.g. requested web page identifier code (such as a 
URL) and user entitlement code. The service provider incorporates 

1 0 the requested web page information in the transport datastream 
transmitted to processor 25 via antenna 10, processor 20 and 
decoder 30. Controller 115 uses the response access data (PIDs of 
the web page data, response program channel number and 
transponder number) to identify and assemble the web page 

1 5 packets in the transport stream as described for the first 

configuration of processor 25. _ 

The PID values of the response web page information 
and corresponding program map information may be 
communicated between service provider and processor 25 in a 

2 0 variety of ways and in a variety of data formats. The PID values 

and program map information may be conveyed, for example m 
accordance with proprietary or custom requirements of a 
particular system such as within MPEG user definable data 
elements or within non-MPEG data such as within the vertical 

2 5 blanking interval (overscan region) of a broadcast television 

signal Further, the PID values allotted to the web information 
may also be transmitted to processor 25 in supplemental program 
guide information. As another alternative, the PID values to be 
used for response data may be determined by a service provider 

3 0 from information transmitted from processor 25. 
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Thi r .i ron fi r ..r ! .ti n n of Professor 25 



In a third configuration, processor 25 both requests 
5 information and receives response information on the same 
communication link e.g. on the telephone link to server 83 or 
connection service 87, via modem 80. In th.s conftguratton 
controller 115 requests Internet web page informat.on by 
hUnl ht ng icon 857 (Figure 2), for example. Whereupon 
10 ontroL U5 (Figure 1) uses request access data JteUp on 
number and URL code and conditional access data) , «o estab^h 
Internet access and request web page data at the URL address yia 
n^dem 80 and server 83. Controller 115 determines whether the 
Tqueld interne, access is authorized from conditional access or 
! 5 mart card User entitlement information as descnbcd ,n 
connection with the second configuration of processor 25 If 
aulrt ed, control* 115 establishes telephone commumcauon 
wi h a s rvice provider via server 83 by dialing a telephone 
number using modem 80 and transmitting t e Interne. URL 
2 0 address code and user entitlement code, for example as 
20 ;tv o'sly described. Controller 115 uses response ^ access ^ da 
(including Transmission Control Protocol/Internet Protocol 
TCP/IP) identification data) determined from the compos.te 
plogram map information to identify and assemble the web page 

2 5 TCP P packets containing the web page HTML and assoc.a ed 

TPFn GTF T jp t yp e data) in the 
JAVA data (and other data e.g. JPECj, uir, l ^ 

rpsnonse datastream from server 83. 

response wefe page TCp/IP packe ts are buffered 

„^»m 80 and provided to HTML™ processor 60 under 
30 :^ZT^nl Processor 60 decodes the HTML™ and 

3 "encoded web response information and prov.des web pa e 
mage representative pixel data for storage m memory wuhtn 
controller 115. Controller 115 retrieves the web page tmage p.xel 
daU from internal memory and formats it for storage as an 

35 o er 1 ay in the pixel representative memory contatne .n display 
nrocessor 70 for reproduction by unit 75 as a merged display, as 
p^us V described In this configuration, processor 25 supports 
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full Internet access via server 83 using the disp ayed progr m 
guide. For example, upon a User highlighting icon 810 followed by 
con 830 (Figure 2) controller 115 (Figure 1) execute, , web page 
browser functions and displays either a custom web page browse 
5 or a standard browser display such as Netscape Navigator™ 
through which full Internet access is available. 

Alternatively, in the third configuration, processor 25 
may support a limited Internet service. In this case access to the 
rnternet'netwo* via server 83 is under the control of a ser 
! 0 provider that allows only indirect Internet access. The sen- ce 
provider provides collated web page informauon for display 
compatible with the displayed composite program guide The 
erX provider stores a limited number of web pages locaUy 
within server 83 that are accessible by processor 25 using the 

1 5 composite program map information which provides an index to 

h T available web pages and their corresponding predetermined 
PIDs Alternatively, the service provider may alter the -lection o 
web pages available from server 83 and periodically provide 
updated compatible supplemental program guide informauon to 

7 0 processor 25 via the satellite broadcast datastream. 

2 0 processor 25^ ^ guide information enables 

processor 25 to form an updated displayed guide together with 
comp ible program map and other access information enabling a 
uTto acces/ the updated web information. By storing ; the ; w * 
25 page Internet information locally in server 83 <h service 
provider retains control of the Internet content available to a Use 
and is able to bill the User for the material prov, ded . Als o the 
local storage of the web page informauon in serve 83 provides a 
User with faster access to the informauon without the full delay 
3 0 Associated with normal Internet access. Further, the bur en o 
processor 25 of communicating informauon requests to externa 
dev c s minimized in that only a limited quantity of nterne 
information may be requested and processor 2 need 
communicate with local servers such as server 83. It .s to be noted 
3 5 haTmodem 80 may a,so be a wide bandwidth commun.cauon 
modem such as a cable modem. In which case, processor 25 may 
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dec ode both web page data and broadcast video program data for 
display from a cable program source via modem 80. 

P Controller 115 employs the method shown in Figure 4 

t0 produce a displayed program guide by P"™^ 
5 formatting received program guide informa 10 

4 does 

incl udes User ^^""^J^L* options and 
not requ.re processor « <o Jorm « conf i g „ration, for 

may be employed by processor proces sor 25 

example. In step 205, following he sta « " t incorporates 

10 receives a transport datastream from , decod r 30 tha P • 

Us er selectable m^u options ^ ^ 

program guide. The User seiecia * telet)ho ne, Internet 

P of communication functions , sue "Jf^^L^g and 
access, fax, video-phone, channel tuning, pr g 

1 5 home control functions. includes additional 

The data received in step aiso mwu 

• a t in the program guide information. The additional 

linking data m the program g conditiona i access and 

25 r^U, 2eJ and display the 

guid e informat ion in ^J^.^ZT^m externa! 
acC ess data enab K l address representative codes 
devices and .nctade d. informatio „ sources. In 

correspondmg to addresses ^ d ata PID 

3 0 step 210, controller 115 uses pr processor 25 to 

values stored in ^«or d -tact-. ^ ^ ^ 
identify and capture the linking and ^ ^ 

program guide information. ™e captur 

. , j f„ rma tted and stored by controller iu 
assembled, formatted Unking data 

35 Interne or Empresses' or telephone numbers may be 

l U „l: by a User via an entry device such as a remote control 
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unit and the additional data is incorporated in the received linking 

data by controller 115. 

The program guide information containing the User 

selectable options, is formatted for display by controller 115 in 
5 conjunction with display processor 70 within processor 25 in step 
220 as previously described in connection with Figure 1. In step 
225* display processor 70 converts the stored and formatted 
program guide data into conventional luminance and chrominance 
components. The luminance and chrominance component 
10 together with timing data provided by controller 115, are 
processed by processor 70 for display on reproduction device 75. 
The process of Figure 4 terminates at step 230. 

Controller 115 (Figure 1) employs the method shown 
in Figure 5 .0 produce a program guide in a video decoder from 
1 5 information received from a plurality of source. The formed 
guide supports User selectable communication functions e.g. 

Fmail fax etc In step 305, following the start at 
Internet access. Email, tax etc. . , cd /-\ 
step 300, controller 115 acquires satellite program guide (SPG) 
formation from the satellite broadcast transport stream received 
10 bv processor 25 from decoder 30. In step 310, controller 115 
acquires DVD program guide (DPG) information from storage 
dev'e 90 via interface 95. Controller 115 acquires, collates and 
formats the SPG and DPG by accessing and assembling respective 
dir ackets .0 produce SPG and DPG representative pixe data for 

2 5 lage as a composite program guide in the unit 70 pixel ™p a^ 

previously described in connection with Figure 1. The SPG and DPG 
^formation include. Unking and access data ,n the form of 
nrosram map, conditional access and network information. In 
o he Configurations, processor 25 applies the principles detailed 

3 0 herein to form a composite program guide including P= 

guide information derived from a source such as a terrestrial 

8 „.m. hmadcast source, a computer source, a 

broadcast source, a cable broadcast source, v 

radio transmission source and a source accessed via telephone 

UneS ' In step 315, controller 115 creates User selectable 

me nu options for display (for User selection of Internet access^ 
Email, fax etc.) in pixel data form for incorporate in the unit 70 
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pixel memory map. In step 3 H. "controller 115 also creates User 
determinable menu options that enable a User to customize h,s 
program guide to include an icon that performs a specf.c User 
requested function, for example, a User may require an .con . ha 
5 accesses a specific Internet site or results in a display of all 
received messages. Controller 115 responds to commands entered 
via a data entry device e.g. a remote control unit or keyboard m 
creating the User determinable menu option in accordance with 
programmed instructions. . 
10 Controller 115, in step 320, examines the SPG and DPG 

information for data identifier (e.g. PID) omissions and conflicts. In 
step 325, controller 115 re-numbers existing data identifiers and 
allocates new data identifiers as required to produce compatible 
program map information for both the collated program guide 
1 5 information and the associated program content. In step 330, 
controller 115 forms composite program map, conditional access 
and network information for the composite program gu.de m the 
manner described in connection with Figure 1. 

The composite program map .s formed to enable 
2 0 processor 25 to perform the communication functions selectable 
via the composite program guide. For this purpose, the compos.te 
n™ map associates a communication function PID and menu 
opln (e.g. P web page. Email, fax icon) with the different a dress 
representative codes corresponding to the addresses of the 

2 5 different information sources. In step 330, controller 115 also 

incorporates User entered data into the composite program map, 
condition^ access and network information. This data may mclude 
items such as an Email address, a telephone/fax number, an 
Internet web page link address, a credit card number etc. The 

3 0 uTenters thU data to processor 25 via a device such as a remote 

control or keyboard. Controller 115. in step 335, formats the SPG 
and DPG pixel data together with the menu option pixel data in the 
unit 70 pixel map created in step 315 to form a compos.te 

program *^**J££ ^ guide informa tion containing 
the User selectable menu options is formatted for display in step 
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340 and output in step 345 as previously described in connect.cn 
with Figure 1. The process of Figure 5 terminates at step 350. 

Controller 115 (Figure 1) employs the process of Figure 
6 to automatically acquire and collate program guide information 
5 from a plurality of sources to form a composite program guide for 
display. In step 405, following the start at step 400, controller 115 
acquires and stores satellite program guide (SPG) Information 
from the satellite broadcast transport stream received by 
processor 25 from decoder 30. Controller 115, in step 41* 
1 0 automatically initiates communication with DVD storage device 90 
(Fiaure 1), via interface 95. The communication is inifated upon 
power-up of processor 25 in response to P re-stored hardware 
configuration information defining the peripheral devices that are 
attached to processor 25 and the associated communication 
1 5 protocols. This information may be contained within the received 
program guide information, or may be stored within processor 25 
a. manufacture, or may be determined locally by processo, : 25 or 
may be determined through a combination of these methods. The 
local determination of hardware configuration by processor 25 
2 0 may be accomplished either by peripheral interrogation initiated 
by controller 115, or by User data entry of hardware conf.gurauon 

In alternative embodiments, communication may be 
automatically initiated upon conditions such as, a) User command 
2 5 via a program guide option selection, for example, b) periphera 
interrogation initiated by controller U5 on a per.odic or 
internment basis, or c) detection of a new or changed electrical 
connection of processor 25 to a peripheral dev.ce such as DVD 



3 0 

90 



device 90. ^ nf . .^.^ communic ation with DVD unit 



L;oniroiici uj iuiu " ivu 

90 via interface 95 using communication protocol codes, 
identification codes, entitlement codes and condmonal access 
code retrieved from the access data within the received program 
guL information. In step 420, controller 115 directs processor 25 
3 5 fo a quire and assemble , the DVD program guide (DPG) information 
derived from DVD storage device 90 via interface 95 in the 
tZr previously described in connection with Figure 1. In step 
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425 controller 115 collates the SPG and DPG information by theme 
and topic by mapping the SPG and DPG information to 
responding theme and topic categories in a master set a.so as 
described in connection with Figure 1. Further, ,n step 425 
5 collier 115 collates the SPG and DPG information by source and 
a^tes a different color attribute to programs and program icons 
from different sources. In addition, controller 115 creates menu 
options permitting a User to sort by source and color. 

Figure 7 shows a black and white representation of a 
1 0 color Program Guide display format, according to the invention, 
hlwing menu options created by controller 115 permitting a User 
o I2.e programs by source and color attribute. Source .cons 
905-925 are associated with off-air broadcast, cable, ad o, 
. „ , ,nrt storaee (e g. DVD source 90 - Figure 1) respectively. 
! 5 S So^e on 930-950 8 are associated with Email, telephone, 
15 deopho e fax and Internet function sources respective^ Upon 
a User highlighting source icon 905, for example, a program guide 
s displayed showing off-air broadcast programs and associated 
cons Similarly, upon a User highlighting source icon 930, for 
2 0 example a Program guide is displayed showing Emai. icons via 
hi a Us!r may activate menus for initiating compos.tion and 
lending o^ Email Messages and reading and deleting of messages 

reCeiV6d ' Icon 900 is a "rainbow" sort icon showing a spectrum 

2 5 of discrete color blocks represented by shades of gray in the black 

and whhe program guide depiction of Figure 7. The color choices 
of In 900 may alternatively be arranged as a continuous color 
irum or selected range of co.ors in a single or a number -rf 
dis" color blocks. Upon a user highlighting a particula. - color 

3 0 w thfu icon 900, programs from the source attributed wiUi tha 

i ,„lnr are displayed. For example, assuming in step 425 
particular color are aispiaycu. r , nurce device 

controller 115 attributes the color purple to the DVD source devic 
90 *en upon a user highlighting the color purple within icon 900 
Te movies available from DVD device 90 become displayed as a 
the mov.es ava . , collation scheme may be 

35 TedT ^Is t other criteria than source. Color may 

be used as tribute to identify programs by theme or topic, for 
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« HUnlaVof movies available for viewing 

r^^^^xJZ ty highiighting a r co,or 

from all sources ^ gulde data may 

within .con 900, for examp e. i ams 
be collated by atmbutes other th o{ 
5 from different sources may be attributed w 
hading or different icon shapes and geometry 

Continuing with the process of Figure 6, i V 
co „troller 115 forms a composite program gurfe or « J- 
th e coLated SPG and DPG information ^^^o. in step 
10 described in connection with Ls for display 

425, creates associated User select ^ ^ fey 

X, display as exemplified ^^E^ for display as 

Figure 6 — -YlsCgure 1) employs the process of Figure 

User via a selection made from opuon on a p y^ ^ 
g uide. in step 505 following th *ar t at ste pj ^ ^ 
retrieves access data m the form ot V ^ ^ 

. irei code for example, from memory in ic v 

25 r — C of a web page ^JS^^Z^ 
The access data ^^^^^ "»* 

- : associated with a particular 

progra m or progr^ g. d e c= . ^ ^ 

estabiish a request communication UnK with server S by dialing 
the retrieved ^L^L* access data 

515 ; nte Let UrI code using the protocol required for 

such as the Interne! ■ » commu „ication link. The required 

3 5 communication on the request com information . I„ step 

protocol is determined from the program map ^ 
520 controller 115 directs processor 25 to receiv 
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input signal received on a response communication link. Controller 
115 directs processor 25 in identifying and retrieving the 
response web page information from the input signal in steps 525 
and 530. Different methods of retrieving response data from the 
5 input signal are employed by processor 25 in steps 525 and 530. 
The selected method depends on whether or not the response 
communication link is the same link used for the web page data 
request. The methods used in identifying and retrieving the 
response data are similar to those previously described in 
10 connection with the three configurations of processor 25. 
Controller 115 applies the appropriate configuration method in 
identifying the response web page data in the input signal, in step 
525 For this purpose, controller 115 uses data identifiers 
determined from program map information that associates the 
15 identifiers with the User program guide selection. Controller 115, 
in step 535, formats the response web page information for 
display and the Figure 8 process ends at step 540. 

Controller 115 (Figure 1) employs the process of Figure 
9 for decoding both input Internet information and compressed 
2 0 video data to form a composite video output for display. Following 
the start at step 600, controller 115, in step 605, determines 
packet identifiers for User requested Internet data and 
compressed image data from program map information. In step 
607 controller 115 determines whether access to the compressed 

2 5 video data and the requested Internet data is authorized from 

conditional access or smart card User entitlement information as 
previously described in connection with the third configuration of 
processor 25. 

If access is authorized, controller 115, in step 610, 

3 0 identifies the web page TCP/IP packets containing the web page 

data in a response datastream from server 83. Similarly, if access 
is authorized, controller 115, in step 615, identifies the 
compressed video data packets containing a User selected video 
program in an input transport stream from decoder 30 to 
3 5 processor 25. Controller 115 identifies the web page and 
compressed video data packets in a similar manner to that 
described in connection with Figure 1. 
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In step 617, controller 115 directs processor 25 
(Figure 1) to decrypt Internet or compressed video data if it is 
encrypted. Controller 115 in step 620, directs processor 25 in 
decoding the identified Internet web page data using HTML™ 
5 processor 60. Controller 115 in step 625, directs processor 25 in 
decompressing the identified compressed video data using MPEG 
decoder 65. The resultant decoded web page data and 
decompressed video data is merged, formatted and displayed on 
unit 75 using display processor 70 under the direction of 
10 controller 115 in step 630. The process of Figure 9 terminates at 
step 640. 

The architecture of Figure 1 is not exclusive. Other 
architectures may be derived in accordance with the principles of 
the invention to accomplish the same objectives. Further, the 
1 5 functions of the elements of processor 25 of Figure 1 and the 
process steps of Figures 4-6, 8 and 9 may be implemented in 
whole or in part within the programmed instructions of a 
microprocessor. In addition, the principles of the invention apply 
to any form of MPEG or non-MPEG compatible electronic program 
2 0 guide. Further, the principles of the invention apply to 
communication on any form of communication link including via 
cable, infra-red, radio, microwave, and via a computer network. In 
addition, it is to be noted that the formation of the composite 
program guide as described herein may be performed at a location 
2 5 remote from a video receiver and accessed by the receiver via a 
communication network. 



